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INTRODUCTION 

-"''The  data  needed  to  calculate  the  sensitivity  to  data  perturbations 
of  the  solution  and  optimal  value  of  a  mathematical  program  are  available 
as  by-products  of  many  solution  algorithms.  Fiacco  La^'  tfmonst rat ed 
this  is  developing  a  penalty  function  technique  for  approximating  the 
parameter  derivatives  of  a  solution  for  quite  general  perturbed  non¬ 
linear  programs.  Armacost  and  My lander  13-j'^ised  this  to  advantage  in 
making  available  the  routine  calculation  of  sensitivity  information  as 
part  of  a  computer  code  for  the  Sequential  Unconstrained  Minimization 
Technique  (SUMT)#^A^~Thus,  based  on  the  differentiable  stability 
theory  developed  DyFfecco  [8]  and  Implemented  by  Armacost  [1]  and 
Armacost  and  Mylander  [3],  a  number  of  stability  measures,  namely  the 
parameter  gradients  of  the  optimal  value  function  and  the  Karush-Kuhn- 
Tucker  triple,  can  be  represented  and  approximated  using,  for  instance, 
only  data  needed  for  the  SUMT  solution  technique  itself. 

Tight  upper  and  lower  bounds  on  the  directional  derivative  limit 
quotients  of  the  optimal  value  function  for  different  forms  of  nonlinear 
programs  with  data  perturbations  have  appeared  in  Gauvin  and  Dubeau  [13], 
Fiacco  and  Hutzler  [10],  Fiacco  [7],  and  Hutzler  [17].  One  form  of  these 
bounds  is  given  by  a  pair  of  linear  programs,  each  of  whose  feasible 
region  is  the  set  of  Karush-Kuhn-Tucker  multiplier  vectors  associated 
with  a  particular  solution  of  the  nonlinear  program. 

In  this  paper,  we  first  consider  equality  constrained  programs 
and  determine  conditions  under  which  the  optimal  value  directional 
derivative  can  be  calculated  using  only  first-order  information  about 
the  problem  functions.  We  also  give  conditions  under  which  this 
directional  derivative  can  be  estimated  using  the  iterates  of  any 
sequential  solution  technique. 

Next  we  consider  the  more  general  pertur  '•gram  containing 

inequality  as  well  as  equality  constraints.  mixed  interior- 

exterior  penalty  function,  we  show  that,  when  the  parameter  directional 
derivative  of  the  optimal  value  function  exists,  it  can  be  approximated 
or  bounded  above,  depending  on  the  nature  of  the  solution  generated  by 
the  penalty  function  algorithm.  Moreover,  we  establish  the  existence 
of  the  parameter  directional  derivative  of  the  mixed  interior- exterior 
penalty  function  and  obtain  a  representation  of  it. 
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NOTATION  AND  DEFINITIONS 

In  this  paper  we  shall  be  concerned  with  mathematical  programs  of 
the  general  form: 


minx  f(x,e) 


P(e) 


s.t. 


gi(x,e)  >  0  (i  =  1, . . . ,ra) 


hj  (x ,  e)  =  0  (j  =  1, ...,p). 


where  xeE  is  an  n-dimensional  vector  of  decision  variables,  e  is  a 
parameter  vector  in  E  ,  and  the  functions  f,  g^,  and  h ^ ,  unless  specified 
otherwise,  are  once  continuously  differentiable  in  x  and  e.  The  m-vector 
whose  components  are  g^x.e),  i  *  l,...,m,  and  the  p-vector  whose  compo¬ 
nents  are  hj(x,e),  j  =  l,...,p,  will  be  denoted  by  g(x,e)  and  h(x,e), 
respectively. 

Following  usual  conventions,  all  vectors  in  standard  form,  except 

gradient  vectors,  will  be  understood  to  be  column  vectors.  The  gradient, 

with  respect  to  x,  of  a  once  differentiable  real-valued  function 
ii  Ic  X 

f:E  x  E  E  will  be  denoted  Vxf(x,e)  and  is  taken  to  be  the  row 
vector  [9f  (x,  e)/9x^, . . . ,  3f  (x,  e)/3xn  ].  The  transpose  of  a  row  or  column 
vector  y  will  be  denoted  y'.  If  g(x,£)  is  a  vector-valued  function, 
g:En  x  E^  ■+  Em,  whose  components  g^x.e)  are  differentiable  in  x,  then 
V^g(x,£)  will  denote  the  m  x  n  Jacobian  matrix  of  g  whose  ith  row  is 


given  by  ^g^x.e),  i  =  1,. 


m.  The  transpose  of  the  Jacobian 

V  g(x, e)  will  be  denoted  V'g(x,e).  Differentiation  with  respect  to  the 
X  x 

vector  e  is  denoted  in  a  completely  analogous  fashion. 

The  feasible  region  of  problem  P(e)  is  the  set  of  points  xeE 
which  satisfy  the  constraints  and  will  be  denoted  R(e).  Thus 


R(e)  *  {x:  g(x,e)  £0,  h(x,e)  =  0). 


The  set  of  solutions  of  P(e)  will  be  denoted  S(e)  and  is  given  by 


S(e)  -  {xeR(e):  f(x,e) 


f*(e) }, 
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min  f(x)  P2(e) 

s.t.  g(x)  >  elt 
h(x)  -  e2, 

where  e  -  (e1>e2)  is  a  vector  in  Em  x  EP.  The  feasible  regions  and 
solution  sets  associated  with  P^(e)  and  P2(e)  will,  as  for  P(e) ,  be 
denoted  R(e)  and  S(e),  respectively.  The  interpretation  of  this  nota¬ 
tion  should  be  clear  from  the  context  in  which  it  is  applied.  Other 
variations  of  P(e)  will  be  explained  as  they  arise  in  subsequent  dis¬ 
cussion. 

Constraint  qualifications  in  mathematical  programming  are  regularity 
conditions  which  are  generally  imposed  to  ensure  that  the  set  of  Karush- 
Kuhn-Tucker  multipliers  corresponding  to  an  optimal  solution  is  nonempty. 
One  constraint  qualification  which  we  shall  employ  throughout  this 
paper  is  the  well-known  Mangasarian-Fromovitz  Constraint  Qualification 
(MFCQ)  which  holds  at  a  point  xeR(e)  if: 

'V/  n 

i)  there  exists  a  vector  yeE  such  that 

V^g^x.e)  y  >  0  for  ieB(t),  and 

V  h.(x,e)  y  ■  0  for  j  =  l,...,p;  and 
x  J 

ii)  the  gradients  V  h  (x,e),  j  *  l,...,p,  are  linearly  independent. 

x  J 

Another  constraint  qualification  we  shall  have  occasion  to  use  is 
the  linear  independence  condition,  designated  LI,  which  holds  at  a 
point  xeR(e)  if  the  gradients  {V^g^x.e),  ieB(c);  Vxhj(x,e),  j  -  l,...,p} 
are  linearly  independent.  Clearly,  for  equality  constrained  programs, 
MFCQ  and  LI  are  equivalent  regularity  conditions. 


Robinson  [25]  has  shown  the  equivalence  of  MFCQ  and  a  form  of 

local  stability  of  the  solution  set  of  a  system  of  Inequalities. 

Gauvin  [12]  has  shown  that  MFCQ  is  both  necessary  and  sufficient  for 

the  set  K(x, e)  to  be  nonempty,  compact,  and  convex.  In  addition,  Gauvin 

and  Tolle  [14]  established  that  MFCQ  is  preserved  under  right-hand-side 

perturbations.  That  is,  if  MFCQ  holds  at  a  point  x  eS(e  )  of  P_(e),  and 
*  *  L 
if  x  -*■  x  with  x  eS(e  )  where  e  -*■  e  ,  then,  for  n  sufficiently  large, 
n  n  n  n  *  * 

MFCQ  holds  at  x_  with  (p  ,u  )eK(x  .e)  such  that  (ji  ,a>  )  ■*■  (p  ,u>  )  for 
**n**nnn  nn 

some  (p  ,w  )eK(x  ,e  ). 

We  shall  also  have  occasion  to  make  use  of  the  following  compactness 
property  for  point-to-set  maps. 

Definition  1.  A  point-to-set  mapping  $:X  -*■  Y  is  said  to  be  uniformly 
compact  near  e  of  X  if  the  closure  of  the  set  U4>(c),  where  the  union  is 
over  all  c  in  N(e),  is  compact  for  some  neighborhood  N(e)  of  e. 

DIFFERENTIAL  STABILITY  OF  THE  OPTIMAL  VALUE  FUNCTION 

Results  relating  to  the  differential  stability  of  the  optimal 
value  function  have  generally  been  directed  toward  the  existence  of 
its  directional  derivative  or  bounds  on  its  directional  derivative 
limit  quotients.  Before  describing  these  results  and  their  relation¬ 
ship  to  results  in  this  paper,  we  state  several  relevant  definitions. 
Throughout  the  remainder  of  this  work,  we  shall,  without  loss  of 
generality,  focus  attention  on  the  specific  parameter  value  e  =  0  when 
considering  the  program  P(e)  and  its  variations. 

k 

Definition  2.  For  z,  a  unit  vector  in  E  ,  the  directional  derivative 

A 

of  f  (0)  in  the  direction  z  is  defined  to  be: 


V‘«» 


1  im 

e  o+ 


(1) 


provided  that  the  limit  exists. 
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If  the  limit  in  (1)  does  not  exist,  then  attention  is  focused 
on  the  upper  and  lower  directional  derivative  limit  quotients  of  f  (0) , 
which  are  respectively  defined  as  follows: 


D+f*(0)  =  lim  sup  f*(Sz)  -  f*(°) 

2  8  -  0+  6 


(2) 


D'f*(0)  -  llm  inf  f*(g2)  ~  f*(Q) 
2  8  -  0+  6 


(3) 


Clearly  the  limits  in  (2)  and  (3)  always  exist,  though  they  may  not 

* 

be  finite.  Furthermore,  D  f  (0)  exists  if  and  only  if  these  two  limits 

z  *  i  *  _  * 

are  equal,  in  which  case  Dzf  (0)  =  (0)  =  Dzf  (0)  . 

One  of  the  earliest  characterizations  of  the  optimal  value  func¬ 
tion  of  a  mathematical  program  was  provided  by  Danskin  [5,6].  Address¬ 
ing  the  problem  minimize  f(x,e)  subject  to  xeR,  R  some  topological 

Ir 

space,  e  in  E  ,  Danskin  derived  conditions  under  which  the  directional 
* 

derivative  of  f  exists  and  also  determined  its  representation.  In 

particular,  he  showed  that  if  R  is  nonempty  and  compact,  and  if  f  and 

its  partial  derivatives  3f/3e  ,  i  ■  l,...,k,  are  continuous  at  the 

1  * 

point  e  =  0,  then  the  directional  derivative  of  f  (0)  exists  and  is 
given  by 


D/<°>  '  xeS(0)  V<-0>-  <4> 

where  S(0)  =  {x:  x  minimizes  f(x,0)  over  R}. 

Although  Danskin' s  result  has  been  extended  to  other  spaces  and  a 
variety  of  functional  forms,  its  principal  restriction  remains  that  the 
feasible  region,  R,  does  not  vary  with  the  parameter  e.  However,  this 
situation  does  not  prevent  Danskin 's  result  from  being  applied  to  problems 
such  as  P(e)  since  inequality  and  equality  constraints  can  be  "absorbed" 
into  the  objective  function  of  a  program  by  the  use  of  an  appropriate 
auxiliary  function,  e.g.,  Lagrangian  or  penalty  function.  Note  also 
that  it  can  be  applied  directly  to  the  dual  of  a  convex  program  with 
right-hand-side  perturbations. 
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For  the  special  case  in  which  R  is  defined  by  inequalities  of  the 
form  g(x,e)  >  0  and  f  and  -g  are  convex  and  continuously  differentiable 

on  XeEn  for  each  fixed  e,  Hogan  [16]  has  given  conditions  under  which 

*  k 

D  f  (0)  exists  and  is  finite  for  all  zeE  .  For  this  problem,  the 

Z 

additional  conditions  ensure  that  S(0)  is  nonempty  and  bounded  and  that 
g(x,0)  >  0  for  some  xeX.  Under  these  conditions, 


D2£*(°> 


min  max 
xeS(O)  peK(x,0) 


tV£f(x,0) 


V  V£g(x,0)]z. 


(5) 


For  programs  without  equality  constraints,  Rockafellar  [26]  has 

shown  that,  under  certain  second-order  conditions,  the  optimal  value 

function  of  ?£(€)  satisfies  a  stability  of  degree  two,  i.e.,  in  a 

neighborhood  of  e  =  0,  there  exists  a  twice  differentiable  function 

<(>:Em  -*■  E^  with  f  (e)  >  4>(e)  and  f  (0)  =  4>(0)  .  Under  this  stability 

* 

property,  bounds  on  the  directional  derivative  of  f  can  be  derived. 

For  convex  programming  problems  of  the  form  P(e),  Gol'stein  [15]  has 

shown  that  a  saddle  point  condition  is  satisfied  by  the  directional 
* 

derivative  of  f  . 

Recent  investigations  of  Gauvin  and  Tolle  [14]  have  focused  on 
the  stability  of  the  optimal  value  function  for  programs  of  the  form 
P2(c),  whose  equality  and  inequality  constraints  are  subject  to  right- 
hand-side  perturbations.  Assuming  the  Mangasarian-Formovitz  constraint 
qualification  and  the  uniform  compactness  of  R(e),  but  without  requiring 
the  convexity  of  the  problem  functions  or  second-order  conditions,  Gauvin 
and  Tolle  showed  that 


D+f*(0)  < 
z 


inf  max 

xeS(O)  (y,u>)  eK(x,0)\ 


P 

i  z  -  2  <*y 
j-1  3 


rn+ji 


(6) 


and 


(7) 


V  (0)  "  xSfo)  (y,u))m£K(x,0)  Wi2i  “  2T  “j^j/* 

It  is  interesting  to  note  that  in  all  of  the  works  just  mentioned 
the  functional  form  that  results  in  portraying  the  stability  of  the 
optimal  value  function  can  be  expressed  as  an  optimization  of  V£L(x, n,w,0) 
This  observation  is  pursued  in  [7,10,13,17]  to  show  that  the  above  results 
do  indeed  generalize  to  problems  of  the  form  P(e),  where  the  parameter 
can  appear  anywhere  in  the  objective  function  and  in  the  inequality  and 
equality  constraints.  The  conditions  imposed  by  Gauvin  and  Tolle  [14] 
are  shown  to  be  sufficient  to  permit  the  extension  of  (6)  and  (7)  to 
the  more  general  program  P(e) .  The  next  lemma  states  that  extension 
in  a  form  most  suited  to  the  needs  of  this  paper. 

Lemma  3.  If,  for  P(e),  R(0)  is  nonempty  and  MFCQ  holds  at  each 
xeS(O) ,  then  for  any  unit  vector  zeE  , 


V  (0)  -  xeS(O)  (u,w)BS(x,0)  7eL(x,li,u,0)  z‘ 


(8) 


and  if  R(e)  is  uniformly  compact  for  e  near  e  =  0,  then 


_  * 


V  <°>  *  CM.co)  - 


holds  for  some  x  eS(0) . 


(9) 


Lempio  and  Maurer  [21]  have  obtained  similar  bounds  under  analogous 
assumptions  that  are  required  to  handle  general  perturbed  infinite  dimen¬ 
sional  programs  of  the  .form  minimize  f(x,e),  subject  to  xeR^  and 
gCx.eJeR^,  where  R^  and  R2  are  arbitrary  closed  convex  sets.  Auslender 
[4]  has  also  obtained  these  bounds  for  the  right-hand-side  perturbation 
problem  P(e),  extending  the  results  of  Gauvin  and  Tolle  [14]  by  using 
a  weaker  form  of  the  Mangasarian-Fromovitz  constraint  qualification. 

This  allows  him  to  replace  the  differentiability  assumption  on  the 
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objective  and  Inequality  functions  with  the  weaker  requirement  that  they 
be  locally  Lipschitz. 

Although  we  are  focusing  attention  on  programs  for  which  the  spaces 
involved  are  finite  dimensional,  we  note  that  most  of  the  sensitivity 
results  mentioned  above  have  been  extended  to  infinite  dimensional 
programs.  For  example,  Maurer  [22,23)  has  recently  obtained  a  character¬ 
ization  of  the  subgradient  of  the  extremal  value  function  for  the  problem 
P(e),  and  has  applied  his  results  to  a  class  of  optimal  control  problems. 

CALCULATING  STABILITY  BOUNDS  FOR  EQUALITY  CONSTRAINED  PROGRAMS 

Consider  the  equality  constrained  program  P^(e) .  In  [17]  we  have 
shown  that  when  the  feasible  region  is  nonempty  and  uniformly  compact 
near  e  «■  0,  if  the  Jacobian  matrix  7h(x,0)  has  rank  p  for  each  xcS(O) 
then 


Dzf*(0)  "  xcS(O)  V'(x»“lx’0]*0)  z*  (10) 

jt 

for  any  unit  vector  zeE  .  Here,  because  of  the  linear  independence  of 
the  constraint  gradients  at  each  xcS(O),  u>(x,0)  is  the  unique  multiplier 
vector  associated  with  x. 

Now,  to  calculate  D  f  (0)  from  (10) ,  one  must  solve  a  typically 
z 

nonlinear  program.  Two  special  cases  in  which  this  situation  can  be 

avoided  are  when  S(0)  is  a  singleton  and  when,  for  each  unit  vector 

zeE*4,  D  f  (0)  *  -D  f  (0)  .  We  shall  now  show  that  in  the  latter  case, 

*  z  ~z 
(0)  can  be  calculated  explicitly. 

We  first  note  that,  from  (10) 

Dzf*(0)  £  VeL(x,u>[x,0),0)  z 

for  any  xcS(0) ,  so,  substituting  -z  for  z  we  obtain 

-D  f*(0)  £  7  L(x,w[x,0],0)  z. 

**  z  c 
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Thus,  If,  in  addition  to  the  hypotheses  which  guarantee  (10),  we  have 

that  D  f  (0)  =  -D  f  (0) ,  then 
z  -z 

Dzf*(0)  =  V£L(x,o)[x,0],0)  z  (11)  . 

for  any  xeS(O).  Again,  oi(x,0)  is  the  unique  multiplier  vector  associated 

with  x.  All  that  remains  to  determine  D  f  (0)  explicitly  is  to  calculate 

z 

w(x,0)  which,  by  the  first-order  necessary  condition,  must  satisfy 

-u/vxh(x,0)  =  Vxf(x,0).  (12) 

Under  the  assumption  that  Vxh(x,0)  has  full  row  rank  for  each  xeS(0), 
we  see  that  (12)  has  a  unique  solution  given  by 

w(x, 0)  =  -V  h(x,0)  (v'h[x,0]  V  h[x,0])-1  v’f(x.O).  (13) 

X  X  X  X 

Substituting  this  expression  into  (11)  we  see  that  D  f  (0)  is  given  by 
V  f (x, 0)  z  -  V  f (x,0)  (v'h[x,0]  fhlx.O])'1  v'h(x,0)  V  h(x,0)z  (14) 

L  A  A  A  c 

is  * 

for  any  xeS(0),  under  the  condition  that  D  f  (0)  =  -D  f  (0). 

z  -z  * 

If,  on  the  other  hand,  S(0)  is  a  singleton,  i.e.,  if  x  is  the 

unique  solution  of  P, (0) ,  then,  from  (10)  it  follows  that  (14)  is  an 

1  * 

exact  representation  of  D  f  (0)  and  holds  without  the  added  hypothesis 
*  *  z 
that  D  f  (0)  =  -D  f  (0) . 
z  -z 

Suppose  now  that  we  have  a  sequence  of  points  {x^},  given  by  some 
algorithm  for  solving  P^(e),  which  converges  to  x  eS(0) .  Since  our 
problem  functions  are  assumed  to  be  once  continuously  differentiable, 

it 

the  expression  for  Dzf  (0)  given  in  (14)  is  a  continuous  function  of 
x.  Thus  for  k  sufficiently  large,  (14),  when  evaluated  at  x,  ,  provides 

is  is  is  K 

an  estimate  of  D^f  (0)  if  x  is  unique.  If  x  is  not  unique,  then 
(14),  when  evaluated  at  x^  for  k  sufficiently  large,  provides  an 
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estimate  of  D  f*(0)  whenever  D  f*(0)  *  -D  f*(0),  and  an  upper  bound  on 
*  2  *  2  *  ~z 
D  f  (0)  whenever  D  f  (0)  j4  -D  f  (0)  . 
z  z  — z 

PENALTY  FUNCTION  DIRECTIONAL  DERIVATIVES 

Before  developing  similar  results  for  the  more  general  program, 

P(e),  we  briefly  review  the  fundamental  convergence  properties  of  penalty 

* 

functions.  These  properties  are  unployed  below  to  approximate  D^f  (e). 

A  detailed  discussion  of  the  properties  of  penalty  functions  can  be 
found  in  Fiacco  and  McCormick  [11]. 

A  general  form  of  the  mixed  interior-exterior  penalty  function  for 
P(e)  can  be  written 

V(x,e,rk>tk)  =  f  (x,  e)  +  s(rk)  +  P^j^  G(x»£)»  (I5) 

where  I  =  I(g[x,e])  and  G  =  G(’n[x,e]),  and  assuming  the  following: 


i) 

I  is  a  continuous  function  on  R"(e)  =  {x: 

g(x,e)  >  0), 

ii) 

if  (x^l  9  R"(e)  with  x^  -*•  x  and  g^(x  ,  e) 

11m 

one  i  *  l,...,m,  then  k  ^  „  I(xke)  *  +  °°; 

=  0  for  at  least 

iii) 

G  is  continuous  with  G(x,e)  =  0  if  h(x,e) 

otherwise; 

>  0  and  G(x,e)  >  0 

iv) 

r  >  r  >0  implies  s(r  )  >  s(r  )  >  0; 
m  n  *  m  n 

v) 

k^-  rk  =  0  ^p1168  k1?*  ®<*k)  =  0i 

vi) 

0  <  t  <  t  implies  0  <  p(t  )  <  p(t  );  and 
m  n  m  r  n 

vii) 

if  k  -*  +  00  monotonically ,  then  p(tk) 

=  OO. 

It  is  by  now  well  known  that  the  sequential  unconstrained  minimization 
of  the  penalty  function  (15)  can  serve  as  a  means  of  approximating 
local  solutions  of  P(e).  The  conditions  under  which  this  is  so  are 
contained  in  Lemma  4,  which  Is  due  to  Fiacco  and  McCormick  [11]. 

Let  H(e)  ■  (x:  h(x,e)  «  0}. 


Lemma  4.  If , 


a) 

b) 

c) 

d) 

e) 


f,  (i  =  and  -h^  (j  ■  l,...,p)  are  continuous; 

I(x,e),  G(x,e),  s(r)  ,  and  p(t)  satisfy  (i)-(vii)  above; 
a  set,  S'(e)  of  local  minima  of  P(e)  is  a  nonempty  isolated 
compact  set; 

S(e)  Pi  cl(R"le]n  H[e]°)  #  *;  and 

-*■  0  and  t^  -*■  +  each  monotonically  with  t^  >  0, 


then 

i)  there  exists  a  compact  set  S"(e)  with  S'(e)  C  S"(e)°, 

and  for  k  sufficiently  large,  the  unconstrained  minima  of 
V(x,e,rk,tk)  in  R"  O  S’'(£)°  exist  and  every  limit  point  of 
any  sequence  {xk>  of  minimizing  points  is  in  S'(c); 

iiJ  r1?*  s(rk)  "  °» 

iii)  P<ck^  G(xk*e)  "  °*  and 

iv)  k1^- V(xk,E,rk’tk)  " 


Now  consider  the  mathematical  program  P(e),  which  contains  inequal¬ 
ity  constraints  as  well  as  equality  constraints.  Recall  that  for  such 
a  program,  the  general  form  of  the  mixed  interior-exterior  penalty  func¬ 
tion  is 


V(x,e,rk,tk)  -  f (x, e)  +  s(rk>  I(x,e)  +  p(tR)  G(x,e),  (16) 

and  that  one  method  for  solving  P(e)  is  to  solve  the  sequence  of  uncon¬ 
strained  minimization  problems 

minx  V(x,e,rk,tk).  Pk(e) 
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Denoting  by  Vfc(e)  the  optimal  value  function  of  the  program  Pk(e),  we 
now  establish  the  existence  of  the  directional  derivative  Dzv£(£)*  *n 
doing  so,  we  assume,  in  addition  to  the  hypotheses  of  Lemma  4,  that  the 
functions ' I(x, e)  and  G(x,£)  are  once  continuously  differentiable. 

Under  our  hypotheses,  V(x, e,r^,t^)  is  continuously  differentiable, 
V  V  exists,  and  V  and  5  V  are  continuous  in  x  and  e.  In  addition,  by 
Lemma  4,  the  unconstrained  minima  of  Pk(0)  lie  in  R"(0)  f\  S"(0)°  for 
k  sufficiently  large,  where  S"(0)  is  compact.  Thus,  taking  X  *  S"(0) 

in  Danskin's  theorem,  it  follows  that  D  V  (0)  exists  for  all  directions 

k  2  11 

zeE  and  all  indices  k  sufficiently  large.  Furthermore, 


»x<°> 


min 

xeSk(0) 


VEV(x,0,rk,tk)  z. 


(17) 


where  Sk(0)  *•  {xeS"(0):  Vk(0)  >  V(x,0,rk> tk> } .  In  particular,  if  xk 
is  the  unique  minlmizer  of  Vk(0),  then 


d2v*(°)  -  vev<xk,0»rk,tk)  Z* 


As  another  consequence  of  (17)  we  note  that,  if  k  is  sufficiently 
large  and  xk  ■  xit(rjc»tjt>0)  is  a  minimizer  of  Pk(0),  i.e.,  xkeSk(0), 
then,  for  any  unit  vector  zeE^, 

DzV0)  -  7eV(xk>0'rk’tk)  z‘  (18) 

Several  observations  now  follow.  First,  from  (18)  we  see  that 

“D-zV0)  -  VK^^k’V  z»  (19) 

so  that,  combining  (18)  and  (19), 

DA<°>  s  2  s 


Thus,  if  D^V^O)  =  -D_zVk(0)  ^or  some  un^t  vector  z  eEk,  then 

D  *Vk(0)  =  7eV(xk*°*rk*tk)  *  (20) 

z 

for  any  minimizer  x^  of  Pk(0).  If  Dzf  (0)  =  -D_zf*(0)  for  all  unit 
vectors  zeEk,  then  (20)  holds  for  all  direction  vectors  z  and  this 

is 

provides  a  necessary  condition  for  the  existence  of  V^V^O)  • 

If  we  now  take  V(x,0,r^, t^)  to  be  the  logarithmic-quadratic  penalty 
function 


W(x 


m  p 

,0,rk)  -  f(x,0)  -  r.  £  In  g,(x,0)  +  £  h2(x,0),  (21) 

K  i*l  1  zrk  j=i  ■* 


then,  from  (17),  for  k  sufficiently  large. 


W0)  ’  Kcfjo)  V'<*-°-'k>  2 


(22) 


for  any  direction  zeE  . 

Now,  choosing  any  sequence  of  minimizers  {x.  }  with  x.eS.  (0),  we 

^  ^  ”  It  K 

have,  by  Lemma  4,  that  x.  -*■  x  where  x  is  a  minimizer  of  P(0).  Thus, 

^  if  it  ^ 

by  the  continuity  of  VeW,  V£W(xk,0,rk)  -►  ,p  ,w  ,0)  as  k  +  •, 

is  it 

where  the  multipliers  p  and  u  are  given  by 


lim 


i  k  -*■  »  gi(xk,0)  ’ 


i  *  1 | • yD| 


*  lim  h1(xk,Q) 

“J  k  “  rk  ! 


j  *  1 ,  •  •  • ,  p  • 


Clearly,  if  MFCQ  holds  at  each  point  of  S(0),  then  (8)  and  (9) 

*  *  * 

apply.  Thus  7ew(xk»0i rfc)  z>  which  converges  to  V  L(x  ,p  ,ui  ,0)  z. 
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In  general  converges  to  a  value  between  the  bounds  expressed  in  (8) 
and  (9). 

* 

In  the  particular  instance  in  which  D^f  (0)  exists  and  is  given 
by 


V  (0)  =  x^s(0)  v£L(x,w[x,0],u>lx,0],0)  z, 

as  for  instance  when  each  xcS(O)  possesses  a  unique  multiplier  vector, 
w'e  may  conclude  that 

D  f*(0)  £  7  L(x*,u[x*,0],u>[x*,0],0)  z  (23) 

z  e 

for  any  x  cS(0) .  Here,  for  k  sufficiently  large,  7  W(x.,0,r.)  z 
A  £  K  K 

approximates  D  f  (0)  if  equality  holds  in  (23),  as  when  ?(0)  has  a 

*  it 

unique  solution,  or  7£W(x^,0,rjc)  z  is  a  strict  upper  bound  on  Dzf  (0) 
if  equality  does  not  hold  in  (23) . 

Example 

To  demonstrate  a  number  of  the  concepts  just  discussed,  consider 
the  problem 


min  ex^  P(e) 

s.t.  g(x,e)  -  -(xj^  -  e)2  -  (x2  +  2)2  +  4  >  0 
h(x,e)  ■  -x^  +  x2  +  e  ■  0 

The  solution  of  this  program  is  easily  determined  to  be  x* 
with 


+  € 
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* 

x2 


0 

2 


£  >  0 

* 

e  <  0,  and  if  £  ■  0,  can  be  any  value 
in  the  interval  [0,2]. 


(24) 


The  optimal  value  function  is 


f*(e) 


e2  e  >  0 

e2  +  2e  e  <  0 


(25) 


We  see  that  f  is  a  continuous  function  of  the  parameter  £»  but  it  is 
not  differentiable  at  e  ■  0.  This  function  does,  however,  have  direc¬ 
tional  derivatives  at  e  ■  0  which  are  given  by 


V*<°> 


0  z  *  1 

-2  z  -  -1 


(26) 


When  e  +  0,  the  derivative  of  f  (e)  is 


2e 

e  >  0 

2c  +  2 

c  <  0 

(27) 


The  logarithmic-quadratic  penalty  function  for  P(e)  is 

W(x, e,r)  *  ex^  -  r  ^[-(x^  -  e)2  -  (x2  +  2)2  +  4]  +  "J"  +  x2  +  e^’ 

Choosing  a  sequence  {r^}  of  positive  real  numbers  with  r^  -*•  0  monotonically , 
we  have  solved  the  sequence  of  minimization  problems 

minx  W(x,c,rk>  Pfc(e) 
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using  the  Sequential  Unconstrained  Minimization  Technique  (SUMT)  developed 
by  Fiacco  and  McCormick  [11]  and  implemented  by  Mylander,  Holmes,  and 
McCormick  (24].  The  results  of  the  minimization  of  the  sequence  of 
problems  Pk(e)  for  £  *  -1  are  shown  in  Table  1. 

The  sequence  of  points,  xfc  -  xk(rk>e),  generated  by  SUMT  can  be 
used  to  bound  the  directional  derivative  Dzf*(e)  as  in  (23),  since 
xkeSk(£)  and,  as  we  shall  see,  almost  each  xeS(c)  possesses  a  unique 
multiplier.  To  this  end,  consider  the  gradient  of  the  Lagrangian  for 
P(e),  i.e.. 


VL(x,  y  ,uj, e)  -  (e  +  -  e]  -  u>,  2y[x2  -  2]  +  w).  (28) 

A  first-order  necessary  condition  that  x*  *  (x*,x*)  be  a  local  minimum 

of  P(e)  is  that  there  exist  vectors  y(x  ,e)  and  w(x  ,e)  such  that 

VL(x  ,y(x  ,e),w(x  , e) , e)  ■  0.  From  (28)  we  see  that  y  ■  y(x  ,e)  and 
*  * 

at  *  u)(x  ,e)  must  satisfy  the  system  of  equations 


£ 

r-  -* 

“  " 

2(xx  -  e)  -1 

u 

-e 

* 

- 

2(x2  -  2)  1 

(i) 

0 

(29) 


Recalling  that  at  a  solution  of  P(c),  x^  ■  x2  +  e,  it  is  easily  shown  that 
the  system  in  (29)  has  a  unique  solution  for  any  value  of  e  ^  0.  Further¬ 
more,  if  e  ■  0,  the  system  has  a  unique  solution  provided  x*  j*  1.  In 
those  instances  when  the  multiplier  is  unique,  it  is  given  by 


"  * 
u 

1 

-e 

* 

Ul  _ 

4(x2  -  1) 

-2e(x*  -  2) 

Thus,  for  the  particular  situation  in  which  e  ■  -1,  we  can  bound 

* 

Dzf  (”1)  as  in  (23).  As  indicated  in  the  discussion  following  (23), 


we  can  approximate  the  bounding  term  algorithmically  by  V^WCx^.-l.r^)  z, 
tor  k  sufficiently  large.  This  is  demonstrated  using  the  figures 
provided  in  Table  1  and  the  formula  for  V^WCx^j-l.r^)  which  is 


VeW(xk*~1*rk)  *  *k 


1  2(xk  +  lj  1  .  .  2  . 

-  r  — T  +7h(xk,-l). 

g(xk,-l) 


The  results  of  this  sequence  of  calculations  are  shown  in  Table  2.  As 
we  can  see,  the  sequence  of  values  V  W(x  ,-l,r.)  is  approximately  zero 
for  k  ■  8.  Thus  we  may  conclude,  from  the  discussion  following  (23), 
that  D  f  (0)  <  0  for  each  ze(-l,l).  This  agrees  with  the  exact  calcula- 

z  it  i, 

tion  of  D^f  (0)  implied  by  (27)  ~il^ce  f  is  actually  differentiable  at 
e  -  -1. 

Unfortunately  this  cannot  be  used  to  bound  the  directional 

* 

derivative  of  f  when  c  •*  'S*  *yw  reason  is  that  not  all  solutions  of 
P(0)  have  a  unique  mulli;  Julias  Sac  tor.  We  have  seen  from  (29)  that  when 
e  *  0,  the  solution  x  *•  [1,11  multiple,  in  fact  infinitely  many, 
multipliers  associated  witJ*  it. 


RELATED  RESULTS 

The  sensitivity  of  the  optimal  value  function  of  a  mathematical 
program  has  been  the  subject  of  a  significant  amount  of  research  in 
recent  years.  Conditions  ranging  from  the  semi-continuity  of  problem 
functions  to  the  assumption  of  strong  second-order  conditions  and 
strict  complementary  slackness  have  resulted  in  a  number  of  static 
measures  of  the  local  behavior  of  the  optimal  value  function.  The 
reader  interested  in  a  comprehensive  review  of  these  results,  as  well 
as  other  results  in  general  mathematical  programming  sensitivity 
analysis,  is  referred  to  the  survey  by  Fiacco  and  Hutzler  [9]. 

Among  the  most  recent  of  these  results  are  those  of  Armacost  and 
Fiacco  [2],  which  show  that  if,  at  a  local  solution  of  P(e),  the 
second-order  sufficiency  condition  holds  along  with  the  linear 
independence  of  the  binding  constraint  gradients  and  the  strict 
complementary  slackness  condition,  then  the  optimal  value  function. 
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f  (e),  is  twice  continuously  differentiable  in  e.  In  addition,  they 

A 

obtained  the  form  of  f  and  its  first  and  second  derivatives  in  terms 

of  the  P(e)  Lagrangian,  its  gradient,  and  its  Hessian,  respectively. 

In  [18],  Jittorntrum  showed  that  the  strict  complementary  slackness 

condition  is  not  essential  to  the  differentiability  of  f  ,  which  results 

from  the  second-order  sufficiency  and  linear  independence  conditions. 

Note  that  under  these  weakened  conditions,  however,  that  V  f  need  not 

e 

be  continuous  and  that  higher  order  derivatives  do  not  necessarily 
exist. 

A 

As  we  mentioned  earlier,  the  differential  stability  of  f  (e)  has 
been  investigated  by  Gauvin  and  Tolle  [14]  for  right-hand-side  perturba¬ 
tions,  and  by  Fiacco  and  Hutzler  [10],  Fiacco  [7],  Hutzler  [17],  and 
Gauvin  and  Dubeau  [13],  for  more  general  programs  of  the  form  P(e).  Each 

of  these  works  produced  bounds  on  the  directional  derivative  limit 
A 

quotients  of  f  .  Moreover,  problems  possessing  special  structure,  e.g., 

convexity,  are  known  to  have  optimal  value  functions  with  an  at  least 

theoretically  computable  directional  derivative.  Danskin  [5,6]  provided 

the  classic  result  in  this  area  in  addressing  unconstrained  problems. 

His  result  readily  applies  to  constrained  problems  since  inequality  and 

equality  constraints  can  be  "absorbed"  into  the  objective  function  of 

a  program  by  employing  an  appropriate  auxiliary  function  (Lagrangian, 

penalty  function,  etc.).  Gol'stein  [15]  and  Hogan  [16],  each  using 

different  assumptions,  demonstrated  the  existence  of  D  f  for  convex 

z 

programs.  Hogan  used  the  form  of  that  derivative  to  develop  a  convergent 
algorithm  for  solving  decomposable  convex  programs.  Aside  from  Hogan's 
application  of  Dzf  ,  it  does  not  appear  that  these  results  have  been 
applied  to  the  development  or  enhancement  of  algorithms  for  solving 
mathematical  programs. 
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